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Abstract This paper describes a multistep 
Enhanced Formative Assessment Program (EFAP) 
that features a Self-Regulated Learning (SRL) com¬ 
ponent The program, which teaches students to 
become more effective learnershas been applied in a 
wide range of academic disciplines. In this paper we 
report on how the EFAP-SRL model can be applied 
to the area of developmental mathematics. In a 
3-year series of studies, EFAP-SRL students enrolled 
in associate degree developmental mathematics 
courses consistently earned higher pass rates in the 
course as well as higher pass rates on the mathemat¬ 
ics portion of the ACT. In addition, there is some 
evidence that program students transferred this 
learninginto subsequent college-level mathematics 
courses. 

The academic readiness of incoming college students 
is a major concern. For example, Tritelli (2003) reports 
that fewer than half of the students who enter college 
directly from high school complete even a minimally 
defined college preparatory program, and, as a result, 
more than half of these entering students are required 
to take at least one developmental course. This lack of 
readiness is further reflected in the lowlevel of success 
in completing the required sequence of developmental 
courses (Bailey, 2009; Hoyt & Sorenson, 2001; Levin 
&Calcagno, 2008). Consequently, it is not surprising 
that only 23% of entering community college students 
earn an associate degree after 6 years (Brock, 2010). 
A major stumbling block and gatekeeper area has 
been mathematics; therefore, this study illustrates 
the implementation and research of the EFAP-SRL 
model in developmental mathematics. 

As research on academic underpreparedness 
becomes more abundant, the situation becomes 
increasingly grim. A recent report from the 
American Diploma Project (Achieve, 2010) found 
that 81% of twelfth graders were rated as having 
below a basic level of performance in Algebra I, 
and 98% of twelfth graders needed preparation 
in Algebra II. This finding is in keeping with The 
Strong American Schools report (2008), which 
found that more than 40% of high school graduates 
entering two-year colleges require mathematics 
remediation at a cost of between 1.85 and 2.35 
billion dollars annually. Lutzer, Rodi, Kirkman, 
and Maxwell (2007) found that developmental 
mathematics courses comprise over half of the 


mathematics course offerings at many two-year 
colleges. Furthermore, the Carnegie Foundation 
(2009) reported between 60% and 70% of develop¬ 
mental mathematics students do not successfully 
complete the prescribed sequence of required 
courses. Notably absent from efforts to improve 
student success is the incorporation of formative 
assessment to improve student self-regulation and 
metacognitive skills. 

Formative Assessment: A Review 
of a Powerful Intervention 

In a series of landmark review articles, Black and 
Wiliam (1998a, 1998b, 2009) dramatically high¬ 
lighted formative assessment’s contribution to 
precollege student learning. They concluded that 
achievement gains generated by using formative 
assessment across a range of content domains were 
among the largest ever reported for education inter¬ 
ventions. Notably, the largest gains were realized 
among low achievers. Black and Wiliam (2009) 
define effective formative assessment as a process 
that involves: (a) teachers making adjustments to 
teaching and learning in response to assessment 
evidence, (b) students receiving feedback about 
their learning with advice on what they can do 
to improve, and (c) students participating in the 
process through self-assessment. 

Definition 

The common denominator in this work is that 
effective formative assessment is an ongoing 
instructional process that systematically incor¬ 
porates assessment, as opposed to calling for a 
particular kind of assessment instrument or test. 
Heritage (2010) makes this a central theme in her 
report when she writes about the risk of 

losing the promise of formative assessment 
for teaching and learning. The core problem 
lies in the false, but none-the-less widespread, 
assumption that formative assessment is a 
particular kind of measurement instrument 
rather than a process that is fundamental and 
indigenous to the practice of teaching and 
learning, (p.l) 

She also goes on to conclude that despite its 
demonstrated effectiveness across a wide vari¬ 
ety of disciplines, formative assessment has not 


2 


JOURNAL o/DEVELOPMENTAL EDUCATION 





been effectively incorporated into college-level 
instruction. 

Feedback and Its Relationship to 
Formative Assessment 

The literature on formative assessment suggests 
that feedback is a key element in assisting the 
learning process for both instructors and students 
(Hattie &Timperley, 2007). Simply put, instructors 
receive feedback from their classroom assessments 
and can use this information to make changes in 
their instructional practices and curricula designs. 
Instructors can also provide feedback to students 
about how they can improve their own learning. 
Students, in turn, are expected to use this feed¬ 
back to make constructive changes in how they 
learn (Black & Wiliam, 2009). Subsequently, the 
formative assessment process is integrated into 
classroom instruction as an ongoing process and 
can promote mastery learning and curriculum- 
based measurement (Fuchs, 1995; Zimmerman 
& DiBenedetto, 2008). 

In this formulation, the formative assess¬ 
ment process not only uses feedback to promote 
content learning, but it also helps students “learn 
how to learn.” Students begin to understand 
their intended learning goals, develop the skills 
to make judgments about their learning in relation 
to a learning standard or instructional outcome, 
and implement a variety of strategies to regulate 
their learning (Heritage, 2010). As such, formative 
assessment merges with theories of metacogni¬ 
tion in general and self-regulation in particular 
(Nicol & Macfarlane-Dick, 2006). Underscoring 
the centrality of feedback in the learning process, 
Hattie and Timperely (2007) reviewed 196 K-12 
feedback studies and found a positive mean effect 
size of 0.79 for achievement measures—an effect 
greater than students’ socioeconomic background 
and reduced class size. However, they cautioned 
that the effect size varied widely depending upon 
the type of feedback that instructors provided for 
their students. The most significant achievement 
gains involved having students receive information 
about a task and how to do it more effectively. Less 
important types of feedback included praise and 
punishment. 

Within this framework, effectively using for¬ 
mative assessment, with its emphasis on developing 
“learning to learn” skills, differs from the typical 
classroom-based assessments used in many college 
classrooms. Although instructors often give their 
students an occasional quiz or exam in addition 
to the midterm and final exams, they typically 
do not use these assessment opportunities to 
improve their own instructional approaches and 
provide students with constructive feedback and 
suggestions for improving their general learning 
strategies. 


The SRL Component of 
Formative Assessment 

To date, the majority of formative assessment 
interventions have emphasized content compe¬ 
tency to the exclusion of “learning how to learn” 
or metacognitive, self-regulatory strategies. The 
EFAP focuses on the idea that formative assess¬ 
ment related to course content is optimized when 
students’ self-regulatory competencies are also 
explicitly targeted for development during the 
assessment process. Throughout the formative 
assessment cycle, students are taught to develop 
and use self-regulation to better use feedback 
and subsequently optimize learning (Hudesman, 
Zimmerman, & Flugman, 2010; Zimmerman, 
Moylan, Hudesman, White, & Flugman, 2011). 

The SRL approach guiding our work is based 
on models of self-regulated learning developed by 
Zimmerman (2000,2002,2006) and Grant (2003; 
Grant & Green, 2001). It is a psycho-educational 


Students begin to understand 
that learning is directly 
related to experimenting 
with different strategies. 

model characterized by continuous feedback 
cycles. In this approach, each feedback cycle is 
broken down into three main phases. The first 
is a planning phase, in which students conduct 
academic task analyses, choose strategies that 
best address their specific learning challenge, set 
identifiable goals, and make self-efficacy and self- 
evaluation judgments to assess the accuracy of their 
level of understanding and content mastery. Next 
is a practice phase, in which students implement 
their plans, monitor their progress, and make real¬ 
time adjustments to their learning plans. This is 
followed by an evaluation phase, in which students 
assess the strategies’ effectiveness based on teacher 
feedback, build on the successful strategies, and/or 
modify or replace less effective ones. The students’ 
responses from the evaluation phase become the 
basis for subsequent, iterative planning phases in 
ongoing SRL cycles. 

The SRL intervention derives much suc¬ 
cess from its cyclical nature; each time students 
complete a cycle, they acquire more feedback and 
therefore, come closer to achieving their learning 
goals. Students begin to understand that learning 
is directly related to experimenting with different 
strategies, a notable shift from the more common 
notion that achievement is simply a function 
of innate ability or some other external factor 
(Zimmerman, 2002). 


The power of SRL competence is highlighted 
in Zimmerman and Bandura’s classic 1994 study 
in social learning theory. They demonstrated that 
students’ SRL skill levels are more highly correlated 
with their college grade point average than their 
scores on standardized tests such as the SAT. 

The Application of the EFAP-SRL: 
Developmental Mathematics 
Examples 

The four studies described following represent a 
portion of our ongoing research on an EFAP-SRL 
Program which has been iteratively developed, 
implemented, and refined. The EFAP-SRL Program 
focuses on improving the students’ academic 
performance through the development of their 
metacognitive skills. The program includes the 
use of a series of specially formatted assessments 
that are followed by self-reflection and revision 
forms, as well as classroom exercises that empha¬ 
size the constructive use of feedback. The model is 
not content-specific and can be implemented in a 
wide variety of courses; for example, the EFAP-SRL 
Program has been applied to entry-level college 
mathematics courses and other STEM disciplines 
such as electromechanical engineering technology 
(Blank, Hudesman, Morton, Armstrong, Moylan, 
& White, 2007; Hudesman, Crosby, Ziehmke, 
Everson, Isaac, Flugman, & Zimmerman, in 
press; Hudesman, Zimmerman, & Flugman, 2005, 
2010; Zimmerman, Moylan, Hudesman, White, & 
Flugman, 2011). Additionally, in one program the 
EFAP-SRL program was applied to 12 different 
disciplines across five schools. Although the fol¬ 
lowing studies were carried out under a variety of 
programmatic and experimental conditions, they 
are reported in one paper given the consistency of 
positive results that, when taken as a whole, support 
the value of the EFAP-SRL program in assisting 
students with their developmental mathematics 
performance. 

Operational Features of the 
EFAP-SRL Model 

The model consists of five major components that 
are designed to effectively deliver a range of differ¬ 
ent course material. 

1. Instructors administer specially constructed 
quizzes that assess both the students’ academic 
content and SRL competencies. 

2. Instructors review and grade the quizzes to 
provide feedback about both the content and 
SRL competencies that students struggled with; 
instructors also use quiz feedback to adjust their 
instruction. 

3. Students complete a specially constructed self¬ 
reflection and revision form for each incorrectly 
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answered quiz question, which affords them an 
opportunity to reflect on and then improve both 
their academic content and SRL processes that 
were incorrectly applied. 

4. Instructors review the completed self-reflection 
forms to determine the degree to which students 
have mastered the appropriate academic content 
and SRL skills. Based on the instructor’s 
evaluation of their work, students can earn up 
to the total value of the original quiz question. 
Based on the reflection form data, instructors also 
have an additional opportunity to make changes 
to the academic content and SRL topics to be 
covered in upcoming lessons. 

5. Instructors use the feedbackprovidedby the quiz 
and self-reflection form as the basis for ongoing 
class discussions and exercises, during which 
students discuss the relationship between their 
academic content and SRL skills. The students 
develop plans to improve these areas. 

Research Questions 

For the four studies reported here, the general 
research question was whether the introduction 
of an EFAP program that features SRL can improve 
the mathematics performance of students enrolled 
in developmental mathematics as measured by 
course pass rates and pass rates on the mathematics 
portion of the ACT. 

In addition, there were four derivative research 
questions: 

1. Are students enrolled in EFAP-SRL classes more 
likely to pass a developmental mathematics 
course than students enrolled in similar courses 
that did not implement the EFAP? 

2. Are students enrolled in EFAP-SRL develop¬ 
mental mathematics classes more likely to pass 
a postcourse administration of the COMPASS 
portion of the ACT than students enrolled in 
comparison sections of the course? 

3. Are students enrolled in EFAP-SRL develop¬ 
mental mathematics classes more likely to enroll 
in a subsequent college-level mathematics course 
than students enrolled in comparison sections 
of developmental mathematics? 

4. Are students enrolled in EFAP-SRL develop- 
mentalmathematicsclassesmorelikelyto succeed 
in a subsequent college-level mathematics course 
than students enrolled in comparison sections 
of developmental mathematics? 

Method 

The following section describes the use of the EFAP- 
SRL program as it was implemented at an urban 
college of technology. The program was applied 
in the same developmental mathematics course 
during various semesters over a 3-year period. 


Participants and Design 

Two studies were conducted during summer ses¬ 
sions (SI in year 2005 and S2 in years 2008 &2009); 
two more were conducted during the academic 
year (AY1 in year 2005 and AY2 in years 2006 - 
2008). In each of the four studies, we compared 
the academic progress of students enrolled in 
EFAP-SRL developmental mathematics classes 
with the academic progress of students enrolled 
in other comparable sections of the same course. 
However, each study was conducted under different 
experimental conditions. 

The site for all of the studies was an urban 
college of technology with an enrollment of more 
than 15,000 students from over 100 countries. 
Typically, students attending the college have 
reflected similar demographics: almost 90% from 
minority groups; 53% female, and 47% male; 49% 
first in their families to attend college; and more 
than one-third required to enroll in developmen¬ 
tal mathematics courses (T. Cummings, personal 
communication, 2011). 


All of the EFAP-SRL 
program instructors 
attended a training session. 


Assignment of Students to 
Program Conditions 

All students entering the college were required 
to take the prealgebra and algebra portions of the 
COMPASS, developed by ACT (1997), which is used 
as a placement test. After taking this placement test, 
the students reviewed their test results with their 
academic advisor. Any student who did not achieve 
a minimum college-designated cut score (which 
varied from 27-35 in prealgebra and 27-30 in algebra) 
was required to enroll in a noncredit developmental 
mathematics course. In the programs described, the 
students’ assignment to a particular section of either 
the EFAP-SRL or comparison group developmental 
mathematics course was made by the academic advi¬ 
sor based on the individual student’s scheduling 
situation and without any input from the EFAP-SRL 
Program staff 

There were several additional attempts to 
further eliminate any bias in the assignment of 
students to either the EFAP-SRL or comparison 
group sections. In one study (AY2), we were able 
to randomly assign the entire student cohort to 
either the EFAP-SRL or comparison group sections 
of developmental mathematics. 

Furthermore, in the AY2 and the S2 studies, we 
were able to compare mean precourse COMPASS 
scores for the EFAP-SRL and control/comparison 


group students. All of the prescores were found 
to be equivalent for the students assigned to the 
EFAP-SRL and the comparison group sections. 

The Selection of Program Instructors 

EFAP-SRL instructors were chosen from a group of 
voluntary participants. Depending upon the study, 
the comparison group consisted of instructors who 
taught the same course at the same time slot, that is, 
during the summer session or during the academic 
year. In other studies, the comparison group con¬ 
sisted of all the day-session instructors who taught 
developmental mathematics that semester. In one 
study (AY2), the control group instructors were 
volunteers who taught the course at the same time 
and agreed to give the same periodic examinations 
to their students. 

Instructor training and program fidel¬ 
ity. All of the EFAP-SRL program instructors 
attended a training session that involved discus¬ 
sions and workshops on the theory and practice 
of EFAP-SRL. During the semester, instructors 
were observed in the classroom by the program 
staff using a checklist of EFAP-SRL activities; see 
Appendix C (p. 31; Zimmerman et al., 2011) for a 
copy of the observation form. It should be noted 
that the observation items focused on instructor 
strategies that would increase the students’ use 
of “learning how to learn” strategies and not on 
the actual mathematics content. This form was 
designed so that observers could determine the 
frequency of SRL-related instructional behaviors 
during a 1-hour session. The observers indicated the 
frequency of SRL-related behaviors on a five-point 
scale that ranged from (1) never to (5) very often. 

Instructor support. The observation form was 
developed to support the SRL instructors. After 
each class visit, the observers met with the instruc¬ 
tor to review any areas where the instructor’s use 
of specific SRL behaviors needed to be built up. 
The content of the observation checklist remained 
fairly standard throughout the different studies. In 
most of the studies there was no attempt to use the 
form in the comparison group sections. However, 
in one study (AY2), we used the form as a measure 
of program fidelity. In this study, two SRL staffers 
observed both SRL and control group classes three 
times during the semester. The experience levels of 
instructors in both the EFAP-SRL and comparison 
groups varied widely. 

A Description of the Four EFAP-SRL 
Implementations 

Table 1 (p. 6) summarizes the conditions and 
sample sizes of the four EFAP-SRL studies. There 
were a total of 10 EFAP sections with an enrollment 
of 253 students, and a total of 42 comparison sec¬ 
tions with an enrollment of 945 students. 

CONTINUED ON PAGE 6 
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CONTINUED FROM PAGE 4 

The two summer implementations. All sum¬ 
mer session courses were scheduled to meet four 
times a week for 6 weeks, totaling 24 sessions. 
Tutoring services were made available to all of the 
students enrolled in the EFAP-SRL and compari¬ 
son sections of developmental mathematics. It is 
worth noting that traditionally, summer session 
students have an advantage over those students 
who take developmental mathematics courses 
during the academic year since summer session 
students only need to focus their efforts on passing 
one course rather than managing a number of other 
courses simultaneously. 

The two academic-year implementations. All 
academic-year courses met twice a week over a 
15-week semester. As mentioned, students in these 
academic-year developmental mathematics classes 
were also enrolled in other developmental and/or 
college-level course work. 

It should be stressed that all developmental 
mathematics courses were required to cover the 
same course content, sequence of topics, and number 
of classroom hours. Furthermore, the final examina¬ 
tions were mandated by the college’s mathematics 
department and were standard across all sections. 

Materials and Procedures 

The focus of the EFAP-SRL Program was to intro¬ 
duce students to EFAP-SRL procedures that would 
enhance their metacognitive abilities, and, in turn, 
improve their mathematics performance. This 
approach contrasts with the traditional instruc¬ 
tional method that focuses exclusively on teaching 
mathematics content strategies, for example, the 
steps needed to solve a quadratic equation by fac¬ 
toring. By contrast, the EFAP-SRL Program focuses 
on teaching students to better plan, practice, and 
evaluate their “learning how to learn” strategies, 
in addition to traditional academic content strate¬ 
gies. By practicing this model students also have 
the potential to transfer the skills they learned in 
the program to subsequent mathematics classes. 
Additional examples of the EFAP-SRL model are 
described in the section on classroom discussions 
and exercises. 

Mathematics quizzes. Students enrolled in 
the EFAP-SRL course sections completed specially 
formatted quizzes that were administered at least 
once a week during the fall and spring semesters 
and up to twice a week during the summer session. 
Each quiz consisted of five mathematics questions 
and required no more than 15 minutes to admin¬ 
ister. For each of the five mathematics questions, 
students were required to make several metacog¬ 
nitive judgments. For example, when completing 
the top portion of the quiz, students were asked to 
predict their quiz grade and to enter the amount 
of time they spent preparing for the quiz. Once 


they started the quiz, students were asked to read 
each question, but before answering it, they were 
asked to make a self-efficacy judgment to indicate 
how confident they were that they could correctly 
solve the problem. After attempting to solve the 
problem, students were asked to make a second self- 
evaluation judgment, indicating how confident they 
were that they had correctly solved the problem. A 
quiz containing sample mathematics questions and 
formatted with the self-efficacy and self-evaluation 
judgments is illustrated in Appendix A. 

Scoring and providing feedback. Reviewing 
the students’ mathematics content and SRL judg¬ 
ments, instructors assembled information that they 
could use to provide feedback for students. Then, 
after determining the areas students struggled 
with the most, instructors could modify their 
instruction. 

The quiz also provided the instructors with 
information about the relationship between the 
students’ quiz scores (i.e., content competencies) 
and their time management, self-efficacy, and 
self-evaluation judgments (i.e., SRL competencies). 
This information is important because struggling 
students frequently make more optimistic predic¬ 
tions about their knowledge than are warranted 
by their actual quiz scores, indicating that they 
often do not recognize the difference between 
“what they think they know” and “what they 
don’t know” (Tobias & Everson, 2002). As a result 
of this false belief, these students do not feel any 
need to remedy the situation by changing their 
“learning how to learn” behaviors. Therefore, they 
continue a destructive cycle of poor planning and 
poor academic outcomes. Being able to provide 


students with ongoing feedback about the rela¬ 
tionship between their actual performance (i.e., 
quiz score) and their predicted scores and also the 
relationship between their preparation time and 
their self-efficacy and self-evaluations judgments 
is critical to improving the students’ mathematics 
and metacognitive skill sets. 

The SRL math self-reflection and mastery 
learning form. For each incorrectly answered quiz 
question, students were expected to complete a 
separate self-reflection and mastery learning form. 
This form was designed to further assist students 
in assessing the relationship between their content 
knowledge and their ability to use critical SRL 
tools. In the first section of this form, students 
were asked to: (a) compare their predicted quiz 
score and their actual quiz score and explain any 
significant discrepancy; (b) evaluate the accuracy 
of their academic confidence judgments (i.e., their 
self-efficacy and self-evaluation judgments) and 
compare them to their actual quiz score; and (c) 
based on the instructor’s written feedback and/ 
or prior class discussions, indicate which of the 
mathematics strategies were incorrectly applied 
when they attempted to solve the problem. 

In the second section of the EFAP-SRL 
Reflection Form, students again solved the original 
problem and included a written description of the 
specific mathematics strategies and procedures 
involved in their work. Students were also required 
to use these same mathematics strategies to solve 
a similar problem. A sample self-reflection form 
is illustrated in Appendix B (Blank et al., 2007). 

Scoring the self-reflection and mastery learn¬ 
ing form. The EFAP-SRL Reflection and Mastery 


Table 1 

A Description of the Four EFAP-SRL Implementations That Were Conducted at 
an Urban College of Technology 



Program 



Groups 

Designation 

Session 

(N) 

SRL sections (n) 

Comparison/control 
sections (n) 

s*i 

Summer 2005 

(189) 

2(47) 

6 (142) 

S*2 

Summers 2008 & 2009 

(526) 

2(43) 

25 (483) 

AY**1 

Fall 2005 

(284) 

3(62) 

8 (222) 

AY**2 

Fall & Spring 2006-2008 

(199) *** 

3 (101) 

3(98) 


* S indicates that the program was implemented during the summer session. 


** AY indicates that the program was implemented during the academic year. 

*** Random assignment with findings from Institute for Education Sciences (IES). 

There were a total of 10 SRL sections with 253 students and a total of 42 comparison sections with 
a total of 945 students. 
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Learning Form is based on a mastery learning 
approach in which students are given multiple 
opportunities to use feedback to improve perfor¬ 
mance. By completing the form, students had an 
opportunity to demonstrate the degree to which 
they could constructively use feedback to master 
the mathematics and EFAP-SRL skills necessary 
to solve the problem. Students who demonstrated 
a complete mastery on this reflection form could 
earn up to 100% of the original credit for a prob¬ 
lem. Instructors again used information from the 
Reflection and Mastery Learning Form to plan 
lessons that demonstrated the relationship between 
mathematics content and EFAP-SRL competen¬ 
cies. Some examples of these exercises are presented 
in the next section. 

Classroom discussions and exercises. The 

quiz/self-reflection process is considered a major 
classroom priority. Instructors use the collected 
information to engage in ongoing class discussions 
that focus on the relationship between effectively 
learning mathematics content and enhancing 
their self-regulation skills. One example of such an 
activity involves having students create individual 
graphs that illustrate the relationship between their 
SRL judgments and quiz scores. In another exer¬ 
cise, instructors might ask students to compare the 
time they spent preparing for the quiz and their 
quiz grades. These student responses are then listed 
on the board. The results are often an obvious cor¬ 
relation between the students’ preparation time 
and their quiz scores. Students are then asked to 
use the feedback from these exercises to design a 
plan for improving their work. 

Performance Measures 

Over the course of the four studies we employed 
up to four different academic outcome measures. 
The specific combination of measures used in any 
one study was largely dictated by administrative 
and financial constraints: 

1. Passing the developmental mathematics course: 
Students who passed the course (i.e., the course 
examinations) were then eligible to retake the 
COMPASS examination. 

2. Passing the Computer-Adaptive Placement 
Assessment and Support System (COMPASS): 
As previously indicated, students were required 
to retake and pass the COMPASS in order to 
satisfactorily “test out” of the developmental 
mathematics course. 

3. Enrolling in college-level math course: Students 
must have successfully completed the develop- 
mentalmathematicscourseandpassedtherelevant 
sections of the COMPASS then subsequently 
enrolled in a college-level mathematics course. 

4. Passing the college-level mathematics course: 
For the purposes of this study, student success 
was measured by assigning either a pass or fail 


grade at the end of the mathematics course. 
Grading standards in this first college-level 
mathematics course are set by the mathematics 
department and include set weights for class 
quizzes, final examinations, etc. Students who 
earned a grade of D or above were considered to 
have passed the course. A withdrawal or a grade 
of F was considered a failure to complete the 
course. A passing grade served as an indication 
that students were able to successfully transfer 
their developmental mathematics skills into a 
subsequent mathematics course. 

Results 

This section reports the results for the four afore¬ 
mentioned studies (SI, S2, AY1, and AY2). Tables 
2 and 3 summarize the data for the two summer 
studies (SI and S2), and Tables 4 and 5 summarize 
the two academic-year studies (AY1 and AY2). 
Depending on the study, the data reported in 
these tables contain up to six academic measures: 
precourse COMPASS scores, developmental 
mathematics course pass rates, postdevelopmental 
mathematics course COMPASS pass rates, enroll¬ 
ment status in college-level mathematics courses, 
pass rates in the college-level mathematics courses 
for those students who enrolled, and college-level 
mathematics course pass rates for the original 
cohort of developmental mathematics students. 

We were not able to generate all six academic 
measures in each study due to administrative and 
financial constraints. However, Table 6 summa¬ 
rizes the data from the four studies in terms of 
the three most common academic progress mea¬ 
sures: (a) developmental mathematics pass-rates, 
(b) enrollee pass rates in college-level mathematics, 
and (c) overall college-level math course pass rates 


for the original cohort of EFAP-SRL and compari¬ 
son group students. 

Summer Programs Using the 
EFAP-SRL Model 

Summer2005 (Study SI). Table 2 summarizes the 
academic progress measures for students enrolled 
in two EFAP-SRL sections and in six comparison 
sections of developmental mathematics. More stu¬ 
dents enrolled in the EFAP-SRL sections passed 
the course compared to the pass rate of students 
in the comparison group sections. Both groups 
of students were monitored during the Fall 2005 
semester by tracking their enrollment in credit¬ 
bearing mathematics courses. A greater percentage 
from the original cohort of EFAP-SRL students 
enrolled in a college-level mathematics course the 
following semester when compared to the students 
from the original comparison group sections. 
Furthermore, a marginally significant percent¬ 
age of the EFAP-SRL students in the college-level 
course passed when compared to the more limited 
number of students from comparison sections that 
enrolled. Overall, more of the original student 
cohort who enrolled in EFAP-SRL developmental 
mathematics sections during the Summer 2005 
passed a college-level mathematics course at the 
end of the Fall 2005 semester. 

Summers 2008and2009 (Study S2). The data 
in Table 3 (p. 8) report on the academic progress 
measures for students enrolled in two summer 
sections of EFAP-SRL and 16 comparison summer 
sections of developmental mathematics. At the 
start of each summer session, there were no sig¬ 
nificant differences on the prealgebra (arithmetic) 
section of the COMPASS exam scores for the two 
groups or on the algebra section of the COMPASS. 


Table 2 

Descriptive Statistics, Percentages; and Chi Square Tests for Dependent Measures by 
Groups in Developmental Mathematics for Summer2005 (Study SI) 





Groups 




SRL EFAP (A/=47) 

Comparison (A/=142) 



Academic progress measures 

% 

(n) 

% 

(n) 

x 2 

P 

Passed developmental math 

78 

(37) 

49 

(70) 

12.69 

.002 

Enrolled in credit math 

77 

(36) 

39 

(56) 

18.05 

.001 

Passed credit math (math enrollees) 1 

83 

(29) 

63 

(35) 

3.35 

.07 

Passed credit math (original cohort) 2 

62 

(29) 2 

25 

(35) 

20.02 

.001 

1 Students who enrolled in the college-level mathematics < 

course 

and passed it. 



2 Those students from the original cohort that passed the college 

:-level math course. 
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More students in the EFAP-SRL sections 
passed the developmental mathematics course 
when compared to the pass rate of students 
enrolled in the comparison sections. Furthermore, 
the EFAP-SRL students passed the COMPASS 
at a higher rate than students enrolled in the 
comparison group. It should be noted that a few 
students (less than 10%) from both groups, who 
failed the COMPASS at the end of the course, 
retook the test after a short “express” course that 
took place directly after the regular summer ses¬ 
sion. Therefore, slightly more EFAP-SRL students 
enrolled in the college-level mathematics course 
than originally passed the COMPASS (see Table 3). 

In the subsequent semester (i.e., Fall 2008 
and Fall 2009) EFAP-SRL students were more 
likely than comparison group students to enroll 
in college-level mathematics courses. There was 
no difference in the pass rate for EFAP-SRL and 
comparison students in the credit-level mathemat¬ 
ics course. More students who enrolled in the 
EFAP-SRL summer programs during 2008 and 
2009 passed a college-level mathematics course 
by the end of the fall semester. 

Academic Year Programs Using the 
EFAP-SRL Model 

Fall2005 (Study AY 1). Table 4 displays the academic 
program data for EFAP-SRL students enrolled in 
two sections and students enrolled in eight com¬ 
parison sections of developmental mathematics. 
More students enrolled in the EFAP-SRL sections 
passed the developmental mathematics course 
than did students enrolled in the comparison sec¬ 
tions. Similarly, more EFAP-SRL students than 
comparison group students enrolled in a college- 
level mathematics course during the following 
semester. There were no significant differences in 
the pass rates in the credit-level math course for the 
EFAP-SRL students and the more limited number 
comparison group students. Overall, more EFAP- 
SRL students who started the EFAP-SRL program 
in the Fall 2005 semester successfully completed 
a college-level mathematics course by the end of 
the next semester (Spring, 2006). 

Academicyears2006-2008 (Study AY2). Table 
5 (p. 10) reports on a 2-year study funded by the 
Institute for Education Sciences (IES). It involved 
students who were randomly assigned to either one 
of three EFAP-SRL sections or one of three control 
group sections. 

Zimmerman et al. (2011) reported no dif¬ 
ferences in the presemester COMPASS scores for 
the two groups. However, the EFAP-SRL group 
achieved higher mean scores on the last two of 
three periodic examinations (e.g., M= 70.64 
and M = 62.10 for the EFAP-SRL and control 
group respectively for periodic examination two, 
F = 4.96 ,p <.05, and M = 68.06 and M = 54.05 for 


the EFAP-SRL and control groups respectively for 
periodic examination three, F = 10.26, p < .05). 
Similarly, the EFAP-SRL group earned a higher 
mean score on the uniform departmental final 
(M = 73.18 and M = 58.03 for the EFAP-SRL and 
control groups respectively, F = 9.96, p < .05). All 


Passed developmental math & 

COMPASS 

Enrolled in credit math 52 

Passed credit math (math enrollees) 72 

Passed credit math (original cohort) 37 


of these examinations were independently scored 
by another member of the mathematics faculty. 

As part of this program, the EFAP-SRL staff 
was trained to make classroom observations using 
the observation form, see Appendix C. Inter-rater 
reliability between observers was moderate (.59). 

CONTINUED ON PAGE 10 


(37) 

23 

(51) 

28.84 

.01 

(32) 

21 

(46) 

21.69 

.001 

(23)' 

67 

(31) 

0.00 

n.s. 

(23) 2 

14 

(31) 

15.37 

.001 


1 Students who enrolled in the college-level mathematics course and passed it. 


2 Those students from the original cohort that passed the college-level math course. 


Table 3 

Descriptive Statistics , Percentages; and Chi Square Tests for Academic Progress Measures 
by Groups in Developmental Mathematics: Summer2008 and Summer2009 (Study S2) 

Groups 


SRL EFAP(43) Comparison (483) 


Academic progress measures 

n 

M 

(%) 

n 

M 

(%) 

x 2 

P 

Pre-COMPASS - Arithmetic 

42 

34.45 


453 

35.86 


0.42 

n.s. 

Pre-COMPASS - Algebra 

42 

25.23 


465 

26.52 


1.23 

n.s. 

Passed developmental math 

35 


(81) 

255 


(53) 

13.03 

.001 

Pass rates for the COMPASS 

26 


(61) 

213 


(44) 

4.26 

.039 

Enrolled in credit math course 

28 


(65) 

183 


(38) 

8.32 

.004 

Passed credit math course 
(math enrollees) 1 

21 


(75) 

139 


(76) 

0.06 

n.s. 

Passed credit math (original cohort) 2 

21 


(49) 

139 


(29) 

7.55 

.006 


Note. The differences in n sizes between the pre arithmetic and pre-algebra compared to the total 
number represented in the comparison are due to the differences among students. For example, one 
student may have failed portion of the COMPASS whereas another student may have failed both. 


1 Students who enrolled in the college-level mathematics course and passed it. 

2 Those students from the original cohort that passed the college-level math course. 

Table 4 

Descriptive Statistics , Percentages; and Chi Square Tests for EFAP-SRL and Comparison 
Groups in Developmental Mathematics: Fall2005 (Study AY1) 

Groups 

EFAP-SRL (62) Comparison (222) 

Academic progress measures % (n) % (n) y 2 p 
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CONTINUED FROM PAGE 8 

These observers visited EFAP-SRL and control 
group classrooms three times each semester and 
used the classroom observation form to measure 
the frequency of EFAP-SRL targeted behaviors. 
EFAP-SRL instructors demonstrated greater use 
of several targeted EFAP-SRL behaviors (M = 7.32, 
SD = 2.59) than did control group instructors (M 
= 4.95, SD = 2.09; Zimmerman et al., 2011). 

In addition, Table 5 reports a number of 
benchmark measures for the EFAP-SRL and 
control group students reflected in the data from 
this study (Zimmerman et al., 2011). For example, 
more students in the EFAP-SRL group passed the 
developmental mathematics course than students 
in the control group. Those students who passed 
the course were allowed to retake the COMPASS. 
Almost twice as many students in the EFAP-SRL 
group passed the COMPASS than did students in 
the control group. 

A further analysis of the data, which was not 
included in the original article, indicates that in 
the subsequent semester more EFAP-SRL students 
registered for a college-level mathematics course. 
Of those students who registered for a college-level 
mathematics course, there was no difference in pass 
rates for EFAP-SRL and control group students. 
Overall, more students who initially enrolled in 
EFAP-SRL sections of developmental mathematics 
succeeded in passing a college-level mathematics 
course by the end of the next semester than the 
original control group students. 

Table 6 summarizes student performance for 
three main academic progress measures that were 
common to the four studies described separately in 
Tables 2 through 5. In all four programs, EFAP-SRL 
students had higher pass rates than comparison 
groups in the developmental mathematics courses. 
Similarly, in all four studies, EFAP-SRL students, 
who originally enrolled in developmental math¬ 
ematics courses, were more likely to have passed 
a college-level mathematics course by the end of 
the subsequent semester. In some programs, EFAP- 
SRL students enrolled in credit-level mathemat¬ 
ics courses passed at a higher rate than students 
from comparison group sections; in other studies, 
there was no difference in the pass rates for the two 
groups. However, it should be noted that among 
the students who passed the EFAP-SRL develop¬ 
mental mathematics course were those who were 
originally in an academically weaker position than 
the students from the comparison groups; yet, the 
EFAP-SRL students, as a group, passed the college- 
level course at levels at least equivalent to those 
students in the comparison groups. 

Discussion 

Over a 3-year period, four programs were imple¬ 
mented to apply the EFAP-SRL model to assist 
associate degree developmental mathematics 


students in improving their mathematics achieve¬ 
ment. Research on these programs was carried 
out under a variety of conditions; however, the 
outcomes have been consistently positive. 

EFAP-SRL program students outperformed 
comparison group students on a variety of academic 
progress measures, including better pass rates in 
developmental mathematics courses, and higher 
pass rates on the COMPASS. In some cases, we were 
able to demonstrate that, of the original cohort, 
significantly more EFAP-SRL students successfully 
completed a college-level mathematics course by 
the end of the semester following their develop¬ 
mental mathematics course. In other programs, 
even though the EFAP-SRL groups contained a 
larger percentage of students who were originally 
in a weaker academic situation, EFAP-SRL students 


passed the college-level mathematics course at the 
same rate as comparison group students. Taken in 
its totality, these studies support the value of the 
EFAP-SRL program in assisting associate degree 
students to improve their performance in both 
developmental and college-level mathematics. 

Study Limitations 

As indicated previously, the experience level of 
mathematics instructor varied considerably. 
In addition, each study reported includes design 
limitations due to program development, imple¬ 
mentation, and research being funded at varying 
levels and for different purposes. For example, in 
two studies of technology (AY 1 and SI), COMPASS 
prescore data were not collected. We were, therefore, 

CONTINUED ON PAGE 12 


Table 5 

Descriptive Statistics, Percentages, and Chi Square Tests for Dependent Measures by Groups 
in Developmental Mathematics for Fall and Spring Semesters 2006 - 2008 (Study AY2) 

Groups 


EFAP-SRL (101) Comparison (98) 


Academic progress measures 

% 

(n) 

% 

In) 

X 2 

P 

Passed developmental math 

50 

(50) 

31 

(32) 

5.83 

.05 

Passed the COMPASS 

47 

(47) 

27 

(26) 

8.57 

.01 

Enrolled in credit math 

42 

(42) 

22 

(22) 

8.53 

.01 

Passed college-level math (math enrollees) 1 

60 

(25) 

59 

(13) 


n.s. 

Passed college-level math (original cohort) 2 

25 

(25) 

13 

(13) 

4.25 

.05 


1 Students who enrolled in the college-level mathematics course and passed it. 


2 Those students from the original cohort that passed the college-level math course. 


Table 6 

Descriptive Statistics, Percentages, and Chi Square Tests for Academic Progress by Groups 
in Developmental Mathematics for Studies 1 -4 

Academic program measures 

Percent passed Percent of enrollees Percent of original cohort 

dev. math passing credit math passing credit math 


Program 

Exp. 

Comp. 

Exp. 

Comp. 

Exp. 

Comp. 

Summer 2005 (SI) 

78% * 

49% 

83% 

63% 

62% ** 

25% 

Summer 2008-2009 (S2) 

81% ** 

53% 

75% 

76% 

49% ** 

29% 

Study 3: Fall 2005 (AY1) 

60% ** 

23% 

72% 

67% 

37% ** 

14% 

Study 4: 2006 -2008 (AY2) 

50%* 

31% 

60% 

59% 

25%* 

13% 


* p < .05. **p<.01. 
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forced to infer that there were no initial differences 
between the EFAP-SRL and comparison student 
groups; this inference was based on the fact that 
no pre-COMPASS score differences were found 
in another study (S2) using the same procedures 
for assigning students to conditions. Similarly, in 
another study that was supported by the Institute 
for Educational Sciences (Zimmerman et al., 2011), 
we were able to initiate a randomized controlled 
study that included more rigorous procedures 
including collection of pre-COMPASS scores 
(where no differences were found), assessment of 
program fidelity of implementation, and follow-up 
of EFAP-SRL and comparison students in subse¬ 
quent mathematics courses. 

Implications for Research and 
Practice with the EFAP-SRL Program 

Since the current studies only investigated the 
impact of the program as a whole on students in 
developmental mathematics, future research is 
needed to investigate causal mechanisms within 
the different EFAP-SRL program components and 
its application across different disciplines. This 
evaluation might be accomplished by studying the 
impact of increasing the frequency of quizzes, as 
well as the role of various content and metacogni- 
tive feedback opportunities. This research would 
investigate the program under five conditions: (a) 
students who receive a “business as usual” condi¬ 
tion; (b) students who receive increased quizzes 
only; (c) students who take more quizzes together 
with content-only formative assessment (i.e., the 
self-reflection form only deals with the mathemat¬ 
ics content); (d) students who take more quizzes and 
receive metacognitive-only self-reflection forms; 
and, finally, (e) students who receive more quiz¬ 
zes and both content and metacognitive formative 
assessment (i.e., a self-reflection form or exercise). 

On occasion, new instructors expressed 
concern over the extra work that they and their 
students must fit into an already tight curriculum: 
the extra quizzes, reflection and revision forms, and 
the accompanying classroom exercises. Instructors 
also reported feeling out of their element; they 
told us they are mathematicians, not educational 
psychologists. Therefore, they did not feel comfort¬ 
able engaging students in what they believed to be 
psychological rather than mathematical exercises. 
We believe that some of these instructor concerns 
can be addressed by using technology to make 
the assessment and quiz portion of the program 
more efficient. For example, Hudesman et al. (2011) 
reported on the use of a tablet PC to automate the 
administration and scoring of the EFAP-SRL 
quizzes. This technology allowed students and 
instructors to create individual and class-wide 
summary data tables and graphs that illustrated 
the relationship between mathematics content and 


SRL skills. Students and the instructor reported 
that using the tablet PC made the quiz-taking 
and feedback process more engaging. There is an 
increasing interest in the application of technol¬ 
ogy to increase the efficiency of the SRL model in 
a variety of academic disciplines (e.g., Kitsantas, 
Dabbagh, Huie, & Dass, 2013). 

Another possible direction for improving 
the delivery of the EFP-SRL program would be 
to develop collaborations. Classroom instructors 
could deliver the mathematics portion of the course 
and specially trained tutors or counselors could 
teach the metacognitive portion of the course. 

Conclusion 

The need to improve academic outcomes for stu¬ 
dents enrolled in developmental skills courses has 
been the subject of much discussion and research. 
Most attempts to deal with this issue have been 
focused on the development and delivery of aca¬ 
demic content material (Black & Wiliam, 1998a, 
1998b, 2009). The research we have reported on, 

The tablet PC made the quiz¬ 
taking and feedback process 
more engaging. 


when taken together with an ever growing body 
of other work, has demonstrated the importance 
of integrating formative assessment and meta¬ 
cognition with academic content instruction so 
that students can optimize their learning. In this 
paper we have presented some examples of how this 
approach can be implemented as part of an EFAP- 
SRL Program. Results of implementation to date 
reflect the program’s positive impact on student 
success and retention, making it an important tool 
for instructors to consider integrating into their 
instructional tool box. 
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